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1 Introduction 

This research has been inspired by several papers on processes with inert drift [5l SI O [1] . The 
model involves a "particle" X and an "inert drift" L, neither of which is a Markov process by 
itself, but the vector process {X,L) is Markov. It turns out that for some processes (X, L), the 
stationary measure has the product form; see [1]. The first goal of this note is to give an explicit 
characterization of all processes (X, L) with a finite state space for X and a product form stationary 
distribution — see Theorem 12.11 

The second, more philosophical, goal of this paper is to develop a simple tool that could help 
generate conjectures about stationary distributions for processes with continuous state space and 
inert drift. So far, the only paper containing a rigorous result about the stationary distribution for 
a process with continuous state space and inert drift, [1], was inspired by computer simulations. 
Examples presented in Section [3] lead to a variety of conjectures that would be hard to arrive at 
using pure intuition or computer simulations. 

2 The model 

Let S = {1, 2, . . . , for some integer > 1 and let d > 1 be an integer. We define a continuous 
time Markov process {X{t), L{t)) on 5 x as follows. We associate with each state j ^ S & vector 
e M^, 1 < j < N. Define Lj{t) = fi{{s G [0,t] : X{s) = j}), where is Lebesgue measure, and 



V 



let L{t) = ^j^sVjLj{t). To make the "reinforcement" non-trivial, we assume that at least one of 
Vj^s is not 0. Since L will always belong to the hyperplane spanned by Vj^s, we also assume that 
d = dim(span{fi, . . . , vn})- 

We also select non- negative functions aij{l) which define the Poisson rates of jumps from state 
i to j. The rates depend on I = L{t). We assume that a^j's are right-continuous with left limits. 
Formally speaking, the process {X, L) is defined by its generator A as follows, 

Af{j, I) = Vj ■ Wifij, l) + Y, ad^)[f{i, I) - fU, I)], j = l,...,N,lG M", 
for / : {1,. . . ,iV} X M'^ ^ M. 
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We assume that iX,L) is irreducible in the sense of Harris, i.e., for some open set U CR"^ and 
some Jo € 5, for all (x, G 5 x R*^, we have for some t > 0, 

P((X(t),L(t))G{io}x[7)>0. 

We are interested only in processes satisfying (|14p below. Using that condition, it is easy to check 
Harris irreducibility for each of our models by a direct argument. A standard coupling argument 
shows that Harris irreducibility implies uniqueness of the stationary probability distribution (as- 
suming existence of such). 

The (formal) adjoint of A is given by 

A*g{j,l) = -Vj -VigijJ) + Y.[''^.j{l)g{i,l) - aji{l)g{j,l)], j = 1, . . . , iV, / G M'^. (1) 

We are interested in invariant measures of product form so suppose that g{j,l) = Pjg{l), where 
J2jesPj ~ ^ /r^ 9{^)dl' = 1- We may assume that pj > for all j; otherwise some points in .S 
are never visited. Under these assumptions, ([1]) becomes 

A*g{j, I) = -Pjvj ■ Vg{l) + Y,[piaij{l)g{l) - pjaji{l)g{l)], j = 1, . . . ,N, I eR'^. 

Theorem 2.1. Assume that for every i and j, the function I — > aij{l) is continuous. A probability 
measure pjg{l)djdl is invariant for the process {X, L) if and only if 

- PjVj ■ V<7(0 + Y.^pmj{l)gil) - pjaji{l)gil)] =0, j = 1, . . . , iV, / G M'^. (2) 

Proof. Recall that the state space S for X is finite. Hence := supjg^ \vj\ < oo. Fix arbitrary 
r, G (0,00). It follows that, 

sup aij{l) = a* < 00. 

i,jeS,l€B{0,r+2t,v,) 

Note that we always have \L{t) — L{u)\ < — u|. Hence, if \L{0)\ < r + t^ti* and s,t > 0, s + t < t^, 
then \L{s + t)\ < r + 2t^v^ and, therefore, 

sup ax{u),j{L{u)) < a* < 00. 

jGS,u<s+t 

This implies that the probability of two or more jumps on the interval [s,s + t] is o{t). Assume 
that 1^1 < r + i^f* and t < t*. Then we have the following three estimates. First, 

P{X{t) = j I X(0) = i,L{0) =l) = a,,{l)t + Rjj/t), (3) 

where the remainder R\-^{t) satisfies ^'^Y>i,jeS,ieB(Q,t,V:t) ^ ^^^i^) fo'^ some R^{t) such that 

\\mt^QR^{t)/t = Q. 

Let aii{l) = -Yjj^i^'ijQ')- We have 

P{X{t) = i, L{t) =l + tvi\ X{0) = i, L(0) = = 1 + aiiil)t + R^iit), (4) 
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where the remainder Rfiit) satisfies supjg^ ;gB(o,t,?;*) ^ -^^(*) for some -R^(t) such that 

hmi^o^^(i)A = 0. 
Finally, 

PiX{t) = i, L{t) ^l + tvi\ X(0) = i, L(0) = /) = Rli{t), (5) 

where the remainder Rfi{t) satisfies sup^^^i^^^Q^^^^^^ \R^i{t)\ < R^{t) for some R^{t) such that 
limt^oR^{t)/t = 0. 

Now consider any function f{j,l) with support in 5 x B{0,r). Recall that \L{t) — L{u)\ < 
v^\t — u\. Hence, Ei^if{Xt, Lt) = for t < and |/| > r + 

Suppose that |/o| <r + v^t^, ti G (0,t*) and s E (0,t* — ti). Then 

= Y.I Y.I f{k,r)P{X{h+s) = k,L{h+s)edr\X{t,)=j,L{h)=l) 
X P{X{ti) = j,L{ti) E dl I X(0) = i,L{0) = lo) 

"E / fU,mxih)=j,L{ti)edr\x{o) = i,L{o)=io). 

We combine this formula with ©-([SI) to see that, 

Ei,kJ{Xt^^s,Lt^+s) — Ei^if{Xt^,Lt^) 

= Y. f E (/(^' ^) + 0(s))(a,fc(/)5 + Rl,/s)) 
jes "^^"^ k&s,k^j 

X P{X{ti) = j, L{ti) E dl I X(0) = i, L(0) = Zo) 



+ E / f{j,l + sv,){l + a,,{l)s + Rli{s)) 

X P(X(ti) = j, L(ti) E I X{0) = i, L(0) = lo) 

+E / (/(i,o+o(^))i?i;(^) 

X P{X{ti) = i, L(fi) E dl I X(0) = i, L(0) = /o) 



V / f{j, l)P{Xih) = j, L{ti) E dl I X(0) 



i,L(0) =/o), 
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which can be rewritten as 



Yl I U\h I + sv,) - f{j, l))PiX{ti) = j, L{ti) € dl I X{0) = i, L(0) = k) 
X P{X{ti) =j,L{ti) e dl I X{0) = i,L{0) = lo) 

+ fU, I + svj)Rli{s) + if{j, I) + Ois))Rliis) I PiX{ti) = j, L{ti) G dl I X{0) = i, L{0) = Zq). 



We will analyze the limit 



Note that 



je5 '^'^ \ \ k€S,kytj / 

+ f{j, I + svj)Rli{s) + {f{j, I) + 0{s))Rli{s)^ P{X{ti) = j, L{ti) e dl I X{0) = i, L(0) = /q) 

= 0. 

We also have 

^'^-Y I ^ + ^^i) - /O'' = j, Hh) e dl I X{0) = i, L(0) = lo) 

= E / '^lf{jJ)-VjP{X{h)=j,L{h)edl\X{0) = i,L{0)=lo), 



and 



X = j, L{h) e dZ I XiO) = i, L(0) = lo) 

E / E ^(^' Oajfc(0^'(^(«i) = J, L{h) e I ^(0) = i, m = lo) 



jes-"^ keS 



This implies that 
d 



dt 
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-T:Ei,iof{Xt,Lt) = lira -{Ei^i^^f{Xt^+s,Lti+s) - Ei^i^fiXt^, Lt^)) 



t=ti sio s 



= E / i'^ifU,i)-Vj + y,f{k,l)ajk{l)]p{X{h)=j,L{h)edl\X{0)=i,L{0)=lo) (6) 
jGS-^^'\ kes J 

= V / Af{j, l)P{X{ti) = j, L{ti) G dl I X{0) = i, L(0) = lo) 

= Eij,Af{Xt„Lt,), (7) 

for all i and |/o| < r + v^^t^^. 
We will argue that 

P{{X{ti), L{ti)) G • I X(0) = i, L(0) = ^ L(ti)) G • | X(0) = i, L(0) = Zq), (8) 

weakly when / —> Iq. Let Tk be the time of the k-th. jump of X. We have 

P(ri > t I X(0) = i,L(0) =l) = exp aii(/ + svi)ds^ . 

Since / ciii{l) is continuous, we conclude that 

P{Ti > t I X(0) = i, L(0) = /) ^ P{Ti > t I X(0) = i, L(0) = /q), 
weakly as / — > /q- This and continuity of / aij{l) for every j implies that 

P{{X{Ti),L{Ti))G ■ \X{0)=i,L{0)=l)^P{{X{Ti),L{Ti))e ■ \ X{0) = i, L{0) = lo), 
weakly when I ^ Iq. By the strong Markov property applied at T^'s, we obtain inductively that 

P{{X{Tk),L{Tk)) £ ■ \X{0)=i,L{0) = l)^P{{X{Tk),L{n))e ■ \ X{0) = i, L{0) = lo), 

when I ^ Iq, for every A; > 1. This easily implies ([8]), because the number of jumps is stochastically 
bounded on any finite interval. 

Since (j, /) ^ Vi/(j, /) • Vj + J2keS /(^' ^)^jk{^) is a continuous function, it follows from ([6]) and 
m that lo ^ iEi^iJ{Xt,Lt) 



IS 

t=ti 



is continuous on the set \lo\ < r + v^t^,. 

t=ti 

Recall that Eii^f{Xt,Lt) = for t < and |/o| > r + v^t^. Hence, lo —>■ ^Eiigf{Xt,Lt 

continuous for all ti < t^^ and all values of i. 

Fix some t < and let ut{j, I) = Ejjf{Xt, Lt). We have just shown that for a fixed t <t^: and 
any j, the function I — ut{j,l) is C^. Hence we can apply ([7]) with f{j,l) = ut{j,l) to obtain, 

^E,,if{Xt, Lt) = lim -{E,jf{Xt+s,Lt+s) - E,,if{Xt, Lt)) (9) 
at sio s 

= lim ^{Ej^iUt{Xs,Ls) - ut{j, I)) 
= iAut){j,l). 
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Since sup^ ^ (V//(j, I) ■ vj + J2keS f(^^ l)0'jk{l)) < oo, formula ([6|) shows that 



sup -Ejjf{Xs,Ls) < oo. (10) 

Now assume that ([2]) is true and let iT{dj,dl) = pjg{l)djdl. In view of ([TO]) , we can change the 
order of integration in the following calculation. For < ti < t2 < t*, using Q, 

E^f{X{t2),L{t2))-E^f{X{ti),L{h)) (11) 
= E/ Ej,if{Xt,,Lt,)p,g{l)dl-Y, I E,,if{Xt„Lt,)pjg{l)dl 

rt2 



V/ -^Ej,if{Xs,Ls)dspjg{l)dl 

J^jR'iJu els 

CY.! ^Ej,if{Xs,Ls)pjg{l)dlds 
Jti TTi ■^K'* "-^ 



t2 



V / {Aus){j,l)pjg{l)dlds. 

Let /i(j, /) = Pjg{l)- For a fixed j and s < t,,, the function Us{j, /) = outside a compact set, so we 
can use integration by parts to show that 

V/ {Aus){jJ)pjgmi = y, f Usij,l)iA*h)ij,l)dl. (12) 
We combine this with the previous formula and the assumption that A*h = to see that 

ft2 



E^f{X{t2),L{t2))-E^f{X{h),L{ti))= I'Y. [ 



u,{j,l)iA*h)ij,l)dlds = 0. 



It follows that t — > Et^ f (X (t) , L{t)) is constant for every function f{j,l) with compact support. 
This proves that the distributions of {X{ti), L{ti)) and {X{t2), L(t2)) are identical under vr, for all 
< ti < t2 < U- 

Conversely, assume that ^{dj^dl) = pjg{l)djdl is invariant. Then the left hand side of (jlip is 
zero for all < ti < t2 < t^. This implies that 



for a set of s that is dense on [0, oo). By (|12p . 



V / {Aus){j,l)pjg{l)dl = 



V / n,(j,/)(A*/i)(j,Od^ = 
.^^Jr^ 



(13) 



for a set of s that is dense on [0,oo). Note that lims^QUs{j,l) = /(j, 0- Hence, the collection of 
functions Us{j,l), obtained by taking arbitrary functions f{j,l) with compact support and 
positive reals s dense in [0, oo), is dense in the family of functions with compact support. This 
and ([13]) imply that A*h = 0, that is, Q holds. □ 
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Corollary 2.2. // a probability measure pjg{l)djdl is invariant for the process {X,L) then 

T.P^v,=0. (14) 

Proof. Summing ^ over j, we obtain 

-PjVj ■ ^9{l) = 0, (15) 

for all /. Since g is integrable over M*^, it is standard to show that there exist li,l2, ■ ■ ■ ,ld which 
span W^. Applying to alUi, ^2, • • • , we obtain ([13]). □ 

It will be convenient to use the following notation, 

=Piaij{l) -pjOji^l). (16) 

Note that bij = —bji. 



Corollary 2.3. A probability measure pjg{l)djdl is invariant for the process {X, L) and g{l) is the 
Gaussian density 

5(/) = (2vr)-'^/2exp(-|/|V2), (17) 
if and only if the following equivalent conditions hold, 

Pj^j ■ I + YlPiaijil) - PjajiiD] =0, j = l,...,N, leR'^, (18) 

Pi^j • / + ^ (/) = 0, j = l,...,N, le R'^. (19) 

Proof. If g{l) is the Gaussian density then ^g{l) = —lg{l) and ([2]) is equivalent to (jlSp . Conversely, 
if ([2]) and (jlSp are satisfied then Vg{V) = —lg{l), so g{l) must have the form (117p . □ 

In the rest of the paper we will consider only processes satisfying (jl8p - (|19p . 

Example 2.4. We now present some choices for a^j's. Recall the notation = max(a;,0), x~ = 
— min(x,0), and the fact that — x~ = x. Given Vj^s, pj's and 6jj's which satisfy (fT9ll and the 
condition bij = —bji, we may take 

aij{l) = {bij{l))+ /pi. (20) 

Then 

ajiil) = {bji{l))+/pj = {-b,j{l))+ /pj = (bijil))- /pj, 

so 

Piaij{l) -pjaji{l) = {bij{l))^ - {bij{l)y = bij{l), 

as desired. 
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The above is a special case, in a sense, of the fohowing. Suppose that pj = pi for all i and j. 
Assume that Vj''s and fej^'s satisfy (fT9|) and the condition hij = —bji. Fix some c > and let 

ai (0 = bij{l)exp{cbij{l)) ^^^^ 
exp{cbij{l)) - exp{-cbij{l)) 

It is elementary to check that with this definition, (jl6p is satisfied for all i and j, because bij(l) = 
—bji{l). The formula ()2ip arose naturally in [5]. Note that (j20p (with all pj's equal) is the limit of 
(PT|) as c ^ oo. 



3 Approximation of processes with continuous state space 

This section is contains examples of processes (X, L) with finite state space for X, and conjectures 
concerned with processes with continuous state space. There are no proofs in this section 

First we will consider processes that resemble diffusions with refiection. In these models, the 
"inert drift" is accumulated only at the "boundary" of the domain. 

We will now assume that elements of S are points in a Euclidean space with n < N . We 
denote them S = {xi,X2, . . . ,xn}. In other words, by abuse of notation, we switch from j to Xj. 
We also take Vj G M", i.e. d = n. Moreover, we limit ourselves to functions bij{l) of the form bij ■ I 
for some vector bij G M". Then ()19p becomes 

+ ^12 + ^13 H 1- biN = -Pivi 

-bi2 - + 623 + ■ • • + &2n = -P2V2 (22) 
-blN - b2N - bsN = -PNVN- 

Consider any orthogonal transformation A : M". If {bij,Vj,pj} satisfy ()22p then so do 

{Abij,Avj,pj}. 

Suppose that aij{l) have the form aij ■ I for some aij G M". If {aij,Vj,pj} satisfy (fTH]) then 
so do {Aaij,Avj,pj}. Moreover, the process with parameters {aij,Vj} has the same transition 
probabilities as the one with parameters {Aaij,Avj}. 

Example 3.1. Our first example is a refiected random walk on the interval [0,1]. Let Xj = 
(j — 1)/(A^ — 1) for j = 1,. . . ,N. We will construct a process with all pj's equal to each other, i.e., 
Pj = 1/N. We will take I G , vi = a and vn = —a, for some a = a{N) > 0, and all other Vj = 0, 
so that the "inert drift" L changes only at the endpoints of the interval. We also allow jumps only 
between adjacent points, so bij = for \i — j\ > 1. Then ([22]) yields 

bi2 = -a/N 
-bi2 + &23 = 

-b(N-i)N = a/N. 

Solving this, we obtain bi^i^i^ = —a/N for all i. 

We would like to find a family of semi-discrete models indexed by N that would converge to a 
continuous process with product-form stationary distribution as ^ oo. For 1 < z < A^, we set 
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aj(j_|_i)(/) = A^(l,N) and a(j_|_i)j(i) = Al{1,N). We would like the random walk to have variance 
of order 1 at time 1, for large A^, so we need 



Ar + Al = N\ (23) 
Since = —a/N for all i, Aji and Al have to satisfy 

Al-Ar = al. (24) 



When I is of order 1, we would like to have drift of order 1 at time 1, so we take a = N. Then ([24]) 
becomes 

Al-Ar = Nl. (25) 

Solving ([23]) and gives 

N^ + Nl , - Nl 

Al = ^^, ^« = ^^- 

Unfortunately, A^ and Ar given by the above formula can take negative values — this is not 
allowed because Ojj's have to be positive. However, for every N, the stationary distribution of L is 
standard normal, so I typically takes values of order 1. We are interested in large N so, intuitively 
speaking, Ar and Al are not likely to take negative values. To make this heuristics rigorous, we 
modify the formulas for Ar and Al as follows, 

-\- Nl N'^ - Nl 
Al = 3^V0VM, Ar = VOV(-iV/). (26) 

Let P/v denote the distribution of {X, L) with the above parameters. We conjecture that as ^ oo, 
P/v converge to the distribution of reflected Brownian motion in [0, 1] with inert drift, as defined 
in [5l d] . The stationary distribution for this continuous time process is the product of the uniform 
measure in [0, 1] and the standard normal; see [T]. 

Example 3.2. This example is a semi-discrete approximation to reflected Brownian motion in a 
bounded Euclidean subdomain of M", with inert drift. In this example we proceed in the reversed 
order, starting with bij^s and Ojj's. 

Consider an open bounded connected set D C M". Let K he a (large) integer and let Dk = 
Z'^/K n D, i.e., Dk is the subset of the square lattice with mesh 1/K that is inside D. We assume 
that Dx is connected, i.e., any vertices in Dx are connected by a path in Dx consisting of edges 
of length l/K. We take S = Dk and I e M". 

We will consider nearest neighbor random walk, i.e., we will take aij{l) = for j l/K. 
In analogy to ([26]) . we define 
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aij{l) = —(1 + (xi - Xj) • /) V V i^^(xi - Xj) ■ I. (27) 



Then bij{l) = {K'^ /N){xi — Xj) ■ I. Let us call a point in 5 = Dk an interior point if it has 2n 
neighbors in Dk- We now define Vj^s using (j22p with pj = 1/\Dk\- For all interior points Xj, the 
vector Vj is 0, by symmetry. For all boundary (that is, non-interior) points Xj, the vector Vj is not 
0. 
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Fix D C M" and consider large K. Let Pk denote the distribution of (X, L) constructed in this 
example. We conjecture that as, K ^ oo, Pk converge to the distribution of normally reflected 
Brownian motion in D with inert drift, as defined in [5l I]- If D is then it is known that the 
stationary distribution for this continuous time process is the product of the uniform measure in 
D and the standard Gaussian distribution; see [1]. 



The next two examples are discrete counterparts of processes with continuous state space and 
smooth inert drift. The setting is similar to that in Example 13.21 We consider an open bounded 
connected set D C M". Let K he a (large) integer and let Dk = 11" jK n D, i.e., Dk is the subset 
of the square lattice with mesh XjK that is inside D. We assume that Dk is connected, i.e., any 
vertices in Dk are connected by a path in Dk consisting of edges of length \ jK. We take S = Dk 
and / € M". 

Example 3.3. This example is concerned with a situation when the stationary distribution has 
the form Pjg{l) where pj^s are not necessarily equal. We start with a "potential" V : D ^ M.. 
We will write Vj instead of V{xj). Let pj = cexp(— V^). We need an auxiliary function 

m{Vj-Vi)-pj{Vi-Vj)- 

Note that dij = dji and for a fixed i, we have djj^ 1 when K ^ oo and \i — JkI = ^/K. 
Let aij{l) = for \xi — Xj\ > l/K, and for \xi — Xj\ = 1/K, 

aijil) = ^(2 + d^j{V^ - Vj) + {Xj - X^) ■ I). 



We set 

«^.(o = r"":; , . , (28) 



aij{l)yO ifaji{l)>0, 
[K"^ /2pi){pi + pj){xj — Xi) ■ I otherwise. 



If aji{l) > and aij{l) > then 
bijil) =Piaij{l) -pjaji{l) 



= —{2{pi - Pj) + {pi{Vi - Vj) - pj{Vj - Vi))dij + {pi{xj - Xi) - pj{xi - Xj)) ■ I) 

= -^(Pi - Xi) • I. 

It follows from (j28p that the above formula holds also if aji(l) < or aij{l) < 0. Consider an 
interior point Xj. For (fT9|) to be satisfied, we have to take 

^i = "~ Yl ^iPi+Pj)ixj - Xi). 

\x,-x,\=l/K 

For large K, series expansion shows that 

Vi ^ -VV. 
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Fix D C M" and consider large K. Let Pk denote the distribution of (X, L) constructed in this 
example. We recall the following SDE from [1], 

dYt = -VV{Yt) dt + Stdt + dBt , 
dSt = -VV{Yt) dt , 

where B is standard n-dimensional Brownian motion and V is as above. Let denote the distri- 
bution of (y, S). We conjecture that as K ^ oo, Pk converge to P*. Under mild assumptions on V, 
it is known that the stationary distribution for (Y, S) is the product of the measure exp{—V {x))dx 
and the standard Gaussian distribution; see [l]. 

Example 3.4. We again consider the situation when all pj's are equal, i.e., pj = 1/N. Consider a 
function V : D ^ R. We let aij{l) = for | l/K. If I X^ X j I = l/K, we let 

= ^(1 + {Vj + V){xj - Xi) ■ I). 



We set 



^ [K^{Vj + Vi){xj -Xi)-l otherwise. 
Then bij{l) = {l/N)K'^{Vj + Vi){xj - Xi) ■ I and 

Vj = K^ {Vj + Vi){xj-Xi). 

\x,-Xj\=l/K 

For large K, we have vj ~ —2W. 

Fix D C and consider large K. Let Pk denote the distribution of {X, L) constructed in this 
example. Consider the following SDE, 

dYt = V{Yt)Stdt + dBt , 
dSt = -2VV{Yt) dt , 

where B is standard n-dimensional Brownian motion and V is as above. Let P* denote the dis- 
tribution of {Y,S). We conjecture that as K ^ oo, Pk converge to P*, and that the stationary 
distribution for {Y, S) is the product of the uniform measure on D and the standard Gaussian 
distribution. 



The next example and conjecture are devoted to examples where the inert drift is related to 
the curvature of the state space, in a suitable sense. 

Example 3.5. In this example, we will identify and C. The imaginary unit will be denoted by i, 
as usual. Let S consist of A'^ points on a circle with radius r > 0, xj = r exp{j2Tri/N), j = 1, . . . ,N. 
We assume that the p^-'s are all equal to each other. 
For any pair of adjacent points xj and Xk, we let 

= -^(1 + (xk - Xj) ■ I), 
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and 




N'^ixk — Xj) ■ I otherwise, 



with the other akj{l) = 0. Then = N{xjj^i — Xj) ■ /, and by ([T9|) we have 

Vj = N'^{xj-i - Xj) + N'^{xj+i - Xj) = 2N^{cos{2tt/N) - l)xj. 

Note that Vj — > —Air'^Xj when N oo. 

Let P/v be the distribution of (X, L) constructed above. 

Let C be the circle with radius r > and center 0, and let Ty be the projection of M? onto the 
tangent line to C at y G C Consider the following SDE, 

dYt = TY,{St) dt + dBt , 
dSt = -ATT^Yt dt , 

where Y takes values in C and B is Brownian motion on this circle. Let P=k be the distribution of 
(y, S). We conjecture that as — > cxd, Pjy converge to P*, and that the stationary distribution for 
(Y, S) is the product of the uniform measure on the circle and the standard Gaussian distribution. 

Conjecture 3.6. We propose a generalization of the conjecture stated in the previous example. 
We could start with an explicit discrete approximation, just like in other examples discussed so far. 
The notation would be complicated and the whole procedure would not be illuminating, so we skip 
the approximation and discuss only the continuous model. 

Let S C M" be a smooth (n — l)-dimensional surface, let Ty be the projection of onto the 
tangent space to 5 at y € 5, let n(y) be the inward normal to S at y € S, and let p be the mean 
curvature at y £ S. Consider the following SDE, 

dYt = TY,{St) dt + dBt , 
dSt = cop^^n(Yt) dt , 

where Y takes values in S and B is Brownian motion on this surface. We conjecture that for some 
Co depending only on the dimension n, the stationary distribution for {Y, S) exists, is unique and 
is the product of the uniform measure on S and the standard Gaussian distribution. 



We end with examples of processes that are discrete approximations of continuous-space pro- 
cesses with jumps. It is not hard to construct examples of discrete-space processes that converge 
in distribution to continuous-space processes with jumps. Stable processes are a popular family 
of processes with jumps. These and similar examples of processes with jumps allow for jumps 
of arbitrary size, and this does not mesh well with our model because we assume a finite state 
space for X. Jump processes confined to a bounded domain have been defined (see, e.g., [2]) but 
their structure is not very simple. For these technical reasons, we will present approximations to 
processes similar to the stable process wrapped around a circle. 

In both examples, we will identify M? and C. Let S consist of points on the unit circle 
D, Xj = exp{j2'iri/N),j = 1,. . . ,N. We assume that the pj's are all equal to each other, hence, 
Pj = 1/N. In these examples, L takes values in M, not M?. 
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Example 3.7. Consider a C^-function V : D —>■ R. We write Vj = V{xj). We define 
Aij,k) 

For any pair of points xj and Xk, not necessarily adjacent, we let 



1 if Xj and Xk are adjacent on the unit circle, 
otherwise. 



]u2 1 



neZ 

where a G (0,2). We define 

,n fajfe(/)VO ifafcj(/)>0, 
^ [iV^^Vfc - y,)^(j,A;)/ otherwise. 

Then 

bjkil) = N{Vk - Vj)A{j, k)l 

and by (jl9p we have 

j:A(k,j)=l 

Note that Vk — > Ay(2;) = when N ^ oo and Xfc ^ x. 

Let Pat be the distribution of (X, L) constructed above. Let W{x) = F(e*^) and let {Z, S) 
be a Markov process with the state space M x M and the following transition probabilities. The 
component Z is a jump process with the drift VW{Z)S = W'{Z)S. The jump density for the 
process Z is J2n& \{x - v) + n27r|-i-". We let St = AW{Zs)ds. Let Yt = exp(iZt) and P* be 
the distribution of {Y, S). We conjecture that Pjy as N ^ oo and the process {Y, S) has the 

stationary distribution which is the product of the uniform measure on D and the standard normal 
distribution. The process (Y, S) is a "stable process with index a, with inert drift, wrapped on the 
unit circle." 

Example 3.8. Consider a continuous function V : D ^M. with Jj^ V{x)dx = 0. Recall the notation 
Vj = V{xj). For any pair of points xj and Xk, not necessarily adjacent, we let 



ajkil) 



1 / 1 

N 



(liVk-V,)l + Y.\^k-j)+nN\-'-A, 



where a € (0, 2). We define 

ajk{l) 



ajA:(OVO ifafcj(0>0, 
xjji^k — yj)i otherwise. 

Then bjkil) = {l/N^)iVk - Vj)l and by ([HD we have 

1 v-^ 
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Note that if argXk y when N ^ oo then Vk V{e^'^) — fj^ V{x)dx = y(e*^). 

Let Pat be the distribution of {X, L) constructed above. Let W{x) = V{e'^^) and let (Z, S) be a 
Markov process with the state space RxM and the following transition probabilities. The component 
Z is a jump process with the jump density f{x) = {W{x) — W{y))s — f J2n£zi(^ ~ v) + n27r)~-^~" 
at time t, given {Zt = y,St = s}. We let St = fQW{Zs)ds. Let Yt = exp(iZ() and be the 
distribution of {Y,S). We conjecture that Pjv — > P* as ^ oo and the process iY,S) has the 
stationary distribution which is the product of the uniform measure on D and the standard normal 
distribution. 
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